Composing by Listening: A Computer-Assisted System for Creating Emotional Music

Most people communicate emotion through their voice, facial expressions, and gestures. However, it is assumed that only “experts” can communicate emotions in music. The authors have developed a computer-based system that enables musically untrained users to select relevant acoustic attributes to compose emotional melodies. Nonmusicians (Experiment 1) and musicians (Experiment 3) were progressively presented with pairs of melodies that each differed in an acoustic attribute (e.g., intensity loud vs. soft). For each pair, participants chose the melody that most strongly conveyed a target emotion (anger, fear, happiness, sadness or tenderness). Once all decisions were made, a final melody containing all choices was generated. The system allowed both untrained and trained participants to compose a range of emotional melodies. New listeners successfully decoded the emotional melodies of nonmusicians (Experiment 2) and musicians (Experiment 4). Results indicate that human-computer interaction can facilitate the composition of emotional music by musically untrained and trained individuals. DOI: 10.4018/jse.2012070103 International Journal of Synthetic Emotions, 3(2), 48-67, July-December 2012 49 Copyright © 2012, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited. al., 2009). Even young children are capable of interpreting the emotional qualities of music (Dalla Bella, Peretz, Rousseau, & Gosselin, 2001; Kastner & Crowder, 1990; Terwogt & Van Grinsven, 1991). It is likely that through passive exposure to music, nonmusicians acquire extensive musical knowledge but they lack the capacity to express it verbally (for a review see Bigand & PoulinCharronnat, 2006). This implicit knowledge assists nonmusicians to perceive and appreciate complex musical structures such as tonal and harmonic relationships (Krumhansl, 1990; Bigand & Poulin-Charronnat, 2006; Tillmann, 2005). These findings imply that nonmusicians have considerable knowledge of music that may even extend to composition. The abilities of nonmusicians in compositional tasks have not been evaluated because they lack the necessary technical skills, including performance ability and musical literacy. Musicians have greater technical ability in both the performance of their instrument and explicit knowledge of musical notation that allows them to realize musical ideas, and to focus on aesthetic goals (Lehmann & Gruber, 2006). One way to probe the knowledge of individuals is through the study of human-computer interaction. Human interaction with technology can often reveal implicit creative and cognitive abilities within an individual (Clark & Chalmers, 1998). For example, Coughlan and Johnson (2008) developed compositional software to probe the processes of composition and creativity. Using this software, users were able to identify and define musical patterns, rules, and boundaries. Individuals varied in musical training, with some having no training at all and other having extensive experience as composers, yet all participants could explore musical ideas with the assistance of this software. Thus, technology facilitated the development of musical thought in a way that would have otherwise not been possible. In music pedagogy, the use of technology has also been important in shaping curricula (Cain, 2004). The form that the technology takes has varied – ranging from the use of electronic keyboards to audio recording and sequencing. Research shows that the use of technology and computers in music education facilitates the ability of students to work collaboratively, experiment with new ideas, and reliably edit their work (Mills & Murray, 2000). The goal of this investigation was to describe and evaluate a computer-based system that allows participants with no musical training to compose emotional music. The system was developed such that technical skills are not required to compose emotional music. The method was intended to mirror psychological processes involved in human creativity, as described by the Blind Variation and Selective Retention model of creativity (BVSR) (Campbell, 1960). According to the BVSR theory of creativity, novel ideas develop in a manner analogous to that of evolution by natural selection. Novel ideas emerge from variations and extensions of previously existing knowledge. Individuals must then select the variation most suited to the requirements of the task. Support for the theory is provided by the observation that the behaviours associated with creativity are often probabilistic (Cziko, 1995; Simonton, 1999, 2007, 2011), just as new genetic variations that occur are random. In the same way, our system involves a cumulative selection process that permits participants to gradually create novel musical ideas that communicate emotion. For each intended emotion, we presented listeners with five pairs of quasi-random pitch sequences. The two melodies within each pair differed from one another in one acoustic attribute (e.g., for intensity, loud or soft). For each pair, listeners choose the melody that most strongly conveyed a target emotion, which was one of anger, fear, happiness, sadness and tenderness. Although the system can accommodate any number of acoustic attributes, the attributes considered in the initial instantiation of the system included pitch height (high or low pitch), intensity (loud or soft), tempo (fast or slow), articulation (whether tones are played with no breaks between them, legato, or if there are pauses between each tone, staccato) and modality (major or minor). Modality is the most complex of attributes considered, and refers to 18 more pages are available in the full version of this document, which may be purchased using the "Add to Cart" button on the product's webpage: www.igi-global.com/article/composing-listening-computerassisted-system/70417?camid=4v1 This title is available in InfoSci-Journals, InfoSci-Journal Disciplines Computer Science, Security, and Information Technology. Recommend this product to your librarian: www.igi-global.com/e-resources/libraryrecommendation/?id=2